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Introduction 

>  The  US  Military,  like  all  Corporations  and  other  Federal  Agencies  is  faced 
with  rising  costs 

□  Weapons  systems,  fuel,  personnel  costs  continue  to  drive  budgets 

□  Budgets  also  have  constraints  and  political  pressures 

>  Required  Readiness  is  also  a  continual  challenge 

□  Deployment  on  short  notice  is  more  and  more  common 

□  Iraq,  Afghanistan,  Horn  of  Africa 

□  Not  just  Major  Combat  Ops,  but  many  other  Operations  Other-Than-War, 
including  missions  within  the  US  for  Humanitarian  Assistance  and  Disaster 
Relief 

>  Service's  Ability  to  train  individuals,  small  units,  and  larger  force  packages 
synthetically  is  accepted  and  expanding.. 

>  Similar  capabilities  for  staffs  at  the  Operational  Level  are  available  but  not 
widely  used  even  though  these  staffs  are  more  logically  and  readily  able  to 
train  in  a  completely  synthetic  environment 
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Introduction 

>  The  issues  surrounding  the  lack  of  large  scale  distributed  synthetic 
training  for  Operational  level  staffs  are  primarily: 

□  An  over  reliance  on  some  mixture  of  live,  virtual,  and  constructive 
training, 

□  The  lack  of  exercise  constructs  which  address  the  specific  training 
requirements  and  appropriately  stress  these  higher  level  staffs, 

□  A  lack  of  quantitative  metrics  evaluating  the  performance  of  the  staff  and 
the  major  functions  and  processes  that  lead  to  the  operational  decisions  of 
the  commander, 

□  A  lack  of  a  systematic  method  of  relating  costs  to  the  performance  of  the 
staff,  and  no  consistent,  repeatable  method  of  tracking  these  costs  for 
training  across  the  Services  at  the  Operational  Level, 

□  No  way  to  demonstrate  the  Return-on-Investment  gained  from  the  large 
expenditures  on  infrastructure  (JNTC),  simulations,  and  scenarios  for  past 
or  future  training. 
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Background 

>  Previous  efforts  for  performance 
assessment: 

□  McGinnis  and  Stone,  Star  tin.  Luck,  Tillson, 
Holden,  Spurgin,  Newell  and  Rosenbloom, 
Mayer-Kress  et.  al. 

>  Previous  related  and  focused  efforts  for 
cost  assessment: 

□  Cokins,  Wright,  Oliver,  Nelson  and  Bely 
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Background 

>  Previous  related  and  focused  efforts 
for  Return-on-Investment: 

□  Phillips,  Dust,  Gordon,  McGibbon,  Cross 
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Hypothesis 

>  Training  at  the  Operational  level  is  less 
beneficial  when  live  forces  vice  synthetic 
are  used.  Live  forces  can  be  used  but,  at  the 
Operational  level  due  to  the  complexity  of 
the  scenarios  required  to  train  these  staffs,  a 
synthetic  environment  is  more  cost  effective 
and  provides  a  much  greater  ROI. 


June  2007 


WG22 


Unclassified 


8 


The  Discipline  of  M&S 


TV 

>«✓  AvA  S  CZZ 


The  Research  Objective 

>  To  develop,  test,  validate,  and  deliver 
a  methodology  which  provides  a 
repeatable  and  reliable  means  to  portray 
ROI  to  Operational  Level  Commanders 

□  The  methodology  should  provide  the  ability  for 
Operational  level  staffs  to  assess  performance, 
costs,  and  value  of  training  either  for  routine 
reporting  or  in  support  of  certification,  regardless 
of  training  environment 
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The  Research  Problem 

•  Primary:  No  method  exists  to  reliably 
determine  the  costs  of  training  at  the 
Operational  level,  regardless  of 
environment 

•  Secondary:  No  common  standard  by 
which  to  measure  accomplishment  of 
Joint  training  objectives  exists  (e.g.,  one 
service  says  it  should  take  2  hours, 
another  says  12  hours 
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Approach 

1.  Develop  a  set  of  common  measures  and  a  model  to  assess 
Operational  level  staff  performance  after  training 

2.  Develop  an  effective  model  of  costs  for  live  versus  synthetic 
training 

3.  Use  results  of  1)  and  2)  to  define  a  reliable  tool  to  measure 
ROI  for  the  range  of  operations  and  scenarios  under  which 
Operational  staffs  are  trained 

a)  Rational  tool  to  capture  performance  measures 

b)  Weighted  averages  and  logarithmic  functions  to  develop 
algorithm 

4.  Participate  in  events  and  use  Subject  Matter  Experts  to 
validate  weights  and  developed  algorithm 
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Potential  Capabilities 

>  Measure  the  desired  result  (task  or  capability  per  the 
environment)  in  order  to  predict  the  cost  and  time 
required  to  achieve  desired  training 

>  Decide  when  desired  training  level  has  been 
achieved 

>  Find  what  is  limiting  further  success  by  looking  at 
supporting  factors 

>  Make  a  decision  about  the  relative  benefit  of 
additional  training  for  an  entire  staff  or  just  specific 
functions 
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Potential  Capabilities 


>  Contrast  live  versus  synthetic  training  costs  and 
choose,  if  appropriate,  when  to  use  live 


>  Limit  proficiency  training  to  shortfall  areas  only 

>  Understand  impact  of  technologies,  capability  gaps, 
or  specific  failures  against  either  tasks  or  capabilities 
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Goals 

>  To  develop  an  effective  model  of  costs  for  live  and 
synthetic  training 

>  To  develop  a  set  of  metrics  for  staff  assessment 

>  To  use  the  results  of  both  1)  and  2)  to  define  a  realistic  and 
reliable  tool  to  define  ROl  tor  the  range  of  operations  and 
scenarios  required. 

>  To  furnish  commands  such  as  JFCOM  a  possible  template 
for  exercise  development  which  can  be  used  to  either 
initially  certify  or  re-certify  FCC  or  JTF  staffs. 
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Performance  Assessment 
Mathematical  Basis 

>Two  calculation  methods  will  be  used  to 
assess  performance 

□  The  weighted  averaging  approach 

□  A  logarithmic  function  approach 

>  There  is  a  possibility  that  only  one  or 
the  other  may  produce  results  that 
accurately  reflect  warfighting  readiness 
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Detailed  Methodology  Scheme 
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Detailed  Methodology  Scheme 
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Pictorial  Example 
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ROI  Example 

>  Determine  ROI  for  differing 
environments 

□  ROI  calculation  of  major  interest  Live  vs. 
Synthetic 

v^ROI  for  achieving  deployment  readiness  in 
differing  environments 

■  A  Cost  for  same  performance  measures  but  differing 
environments  and  costs  for  same  measures 

■  Performance  /  A  Cost 
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ROI  Example 

>  Determine  ROI  for  new  JTF  supporting 
capability 


□ROI  calculations  from  the  previous  slides  possible: 

ROI  for  achieving  deployment  performance  with  new 
capability: 

■  A  Cost  for  same  measures  but  differing  capability 

■  Performance/  A  Cost 
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Applicability 

>Non  -  Military  Applications: 

□Federal  agencies,  state  agencies,  municipal 
agencies  also  have  to  train  to  manage 
complex  situations 

v^These  require  defining  Mission  Essential  Tasks 
and  developing  standards  of  performance  to 
predict  success 

Costs  can  also  be  similarly  defined 
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Applicability 

>Non  -  Military  Applications: 

□Non  Government  Agencies  such  as  the 
Red  Cross  or  other  relief  organizations  also 
have  readiness  and  response  criteria 

□Large  corporations  with  Global  concerns 
have  similar  needs  for  crisis  management 
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Conclusions 

>  There  is  a  need  for  a  means  for  the  Combatant 
Commanders,  Operational  level  Commanders,  and 
Services  to  determine  the  Return-on-Investment  for  the 
personnel,  systems,  and  infrastructure  necessary  to 
assure  deployment  readiness  in  a  variety  of  roles. 

>  The  most  readily  available  basis  for  ensuring 
commonality  is  the  use  of  the  Universal  Joint  Task  List 

>  A  method  which  incorporates  widely  used  applications 
provides  greater  utility  and  promotes  greater  usage 
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Conclusions 

>  The  role  of  the  Commander  is  not  being  challenged,  but 
the  activities  of  the  subordinate  staff  can,  in  many  cases, 
be  objectively  evaluated,  providing  a  necessary 
component  to  determining  ROI. 

>  The  Services  are  moving  towards  the  concept  of 
"operational  templates'^for  Senior  staff  evaluation  and 
readiness  reporting,  so  this  effort  could  aid  and 
accelerate  that  process. 

>  The  use  of  large-scale,  distributed  synthetic  training  to 
meet  the  wide  scope  of  operational  needs  and  short 
notice  timelines  will  be  quantifiable  and  should  enhance 
it's  use  as  well  as  provide  a  DoD  wide  basis  for 
continuing  use  and  investment. 
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Questions/Comments? 
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